1. Field of the Invention
The present invention relates to a system and method for modifying brightness variations between image frames in digital video.
2. Description of the Prior Art
There is an increasing reliance on video data in rich media applications running on interconnected devices or systems such as personal computers, wireless devices, surveillance systems, video conferencing systems and set-top boxes. Video data compression systems play a key role in increasing the efficiency of video data transmission. Video data is compressed or coded for transmission by taking advantage of the spatial redundancies within a given frame and the temporal redundancies between successive frames. Intraframe compression operates on single frames independently of other frames to exploit spatial redundancies within the frame, whereas interframe compression exploits both spatial and temporal redundancies.
Video compression systems exploit temporal redundancies using interframe prediction coding. Interframe coding is based on predicting the current frame using the previously coded frame, and coding only the prediction error between the source frame and the predicted frame. Approximations are done in the prediction process, by assuming the motion is uniform across all pixels of each motion estimation block in each frame. It is noted that interceding can be done for both uni-directional and bi-directional prediction. Transmission efficiencies are realised in intercoding by transmitting the prediction error, as the amount of information present in the prediction error is generally less than that in the actual pixel values. However, brightness changes between successive video frames can degrade the efficiency of the intercoding process. The main reason for the performance degradation is that the motion prediction in interceding is based on the assumption that the moving object would have the same luminance values in two successive frames. When the luminance values change due to simply changes in brightness, the encoder interprets the changes as being due to motion. Consequently, the encoder would in that case work harder to estimate the perceived motion which could lead to a slowdown of the performance of the encoder. Moreover, the additional generated motion information could cause an increase in the bit rate and could also lead to the appearance of blocking artifacts. Therefore, brightness changes in successive video frames can have a detrimental effect on a number of video coding and video quality aspects.
For example, brightness changes could be due to changes in the camera exposure adjustment, lighting changes, transition fades, etc. In the case of video conferencing using a webcam, when a subject is very close to the camera, brightness changes occur often if the subject frequently moves close to and then away from the camera or moves across the field of view of the camera. These frequent changes in luminance values in video conferencing applications place an increasing demand on the motion estimation and compensation algorithms of the coding process. Also, given that video conferencing applications usually run at low bit rates, the increase in the number of bits required to code the video images due to the coding of the brightness changes would lead to a reduction in the quality of the video images. The latter is due to the fact that fewer bits would be available to accurately reproduce the details of the video images as compared to the case where no brightness changes take place.
A well-known contrast-change/brightness model has been proposed in the literature to describe brightness changes between successive frames. The model is given byI(x,y,t)=C×I(x+Δx,y+Δy,t−1)+D,where I (x,y,t) is the luminance intensity of pixel (x,y) at source frame t, (Δx, Δy) is a motion vector for the pixel, C is a contrast change, and D is a brightness shift.
The above model has been applied in U.S. Pat. No. 6,266,370 to avoid as much as possible coding the brightness changes at the encoder level while at the same time allowing the decoder to reproduce the brightness changes in the decoded images. At the encoder side, the brightness change model parameters are estimated using the brightness level of each pixel in the source frame. When the estimates for the parameters C and D become available, and before the start of motion estimation, the encoder adjusted the brightness level in the reference frame using the estimated brightness change model to bring the brightness level in the stored reference frame closer to that of the source frame. The motion estimation is then performed on the brightness compensated reference frame. Consequently, the encoder does not have to encode the changes in the source frame that are due to brightness changes. The encoder also provided the decoder with the necessary information to reproduce the brightness level changes in the stored reference frame. The latter is achieved by sending the decoder the values of the parameters C and D that could be used to adjust the brightness level in the stored decoded frame as it was done in the encoder. Therefore, the viewer at the decoder side would still be able to notice the temporal variation in brightness as in the source video images, which is desirable in some applications such as in motion picture related applications, however, in other applications such as video conferencing, it is desirable to not have the temporal variation in brightness so that the viewer at the decoder side would not notice such variations. The method presented in U.S. Pat. No. 6,266,370 aims at preserving the temporal variations in brightness as opposed to maintaining the brightness level in the video images relatively constant.
It is an object of the present invention to provide a global brightness change compensation system and method to obviate or mitigate some of the above-presented disadvantages.